Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system

نویسندگان

  • Kendall Lowrey
  • Svetoslav Kolev
  • Jeremy Dao
  • Aravind Rajeswaran
  • Emanuel Todorov
چکیده

Reinforcement learning has emerged as a promising methodology for training robot controllers. However, most results have been limited to simulation due to the need for a large number of samples and the lack of automated-yet-safe data collection methods. Model-based reinforcement learning methods provide an avenue to circumvent these challenges, but the traditional concern has been the mismatch between the simulator and the real world. Here, we show that control policies learned in simulation can successfully transfer to a physical system, composed of three Phantom robots pushing an object to various desired target positions. We use a modified form of the natural policy gradient algorithm for learning, applied to a carefully identified simulation model. The resulting policies, trained entirely in simulation, work well on the physical system without additional training. In addition, we show that training with an ensemble of models makes the learned policies more robust to modeling errors, thus compensating for difficulties in system identification. The results are illustrated in the accompanying video.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Policy Transfer via Modularity

Non-prehensile manipulation, such as pushing, is an important function for robots to move objects and is sometimes preferred as an alternative to grasping. However, due to unknown frictional forces, pushing has been proven a difficult task for robots. We explore the use of reinforcement learning to train a robot to robustly push an object. In order to deal with the sample complexity of training...

متن کامل

Passive Non-Prehensile Manipulation of a Specific Object on Predictable Helix Path Based on Mechanical Intelligence

Object manipulation techniques in robotics can be categorized in two major groups including manipulation with and without grasp. The aim of this paper is to develop an object manipulation method where in addition to being grasp-less, the manipulation task is done in a passive approach. In this method, linear and angular positions of the object are changed and its manipulation path is controlled...

متن کامل

Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning

Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn th...

متن کامل

Optimal Trajectory Planning of a Box Transporter Mobile Robot

This paper aims to discuss the requirements of safe and smooth trajectory planning of transporter mobile robots to perform non-prehensile object manipulation task. In non-prehensile approach, the robot and the object must keep their grasp-less contact during manipulation task. To this end, dynamic grasp concept is employed for a box manipulation task and corresponding conditions are obtained an...

متن کامل

What Does Physics Bias: A Comparison of Model Priors for Robot Manipulation

We explore robot object manipulation as a Bayesian model-based reinforcement learning problem under a collection of different model priors. Our main contribution is to highlight the limitations of classical non-parametric regression approaches in the context of online learning, and to introduce an alternative approach based on monolithic physical inference. The primary motivation for this line ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2018